Feature-region extraction method and feature-region extraction circuit

ABSTRACT

The present invention is intended to exactly extract a feature-region along its contour line without erroneously extracting any moving object other than the feature-region. An extraction signal from a feature-region main-portion extracting circuit is sent to a small region eliminating circuit whereby an erroneously extracted small region is detected and eliminated as an erroneously extracted noisy portion. The small region eliminating circuit compares a small region with four predetermined detection patterns, discriminates it as a region other than the feature (face)-region if no match is found and causes a center pixel of a window of 3×3 pixels to have a logical value of zero. An output of the small region eliminating circuit is sent to a blank filling circuit which compares its input with four predetermined detection patterns, discriminates it as a face-region if a match is found and makes a central pixel of a window of 13×13 pixels have a logical value of 1. The above-mentioned extraction method can reliably extract features of the face-region and is free from the erroneous extraction of any moving object other than the face.

This application is a continuation of application Ser. No. 08/276,205now abandoned filed on Jul. 18, 1994.

BACKGROUND OF THE INVENTION

The present invention relates to a feature-region extraction method anda feature-region extraction circuit and, more particularly, to afeature-region extraction method and a feature-region extraction circuitwhich are capable of extracting a feature-region of an image forprocessing the image according to its feature at a pre-processingportion of an image coding device and which are intended to be used, forexample, in videotelephone systems, video conference systems and so on.

At a motion picture coding and transmitting device for a videotelephone, since any transmission line has a limited or finite capacityto transmit a number of bits at a time, it cannot allocate enough bitsto an image information. Accordingly, a decoded image may have animpaired picture quality with a mosquito-like distortion, a blockdistortion and the like. On the other hand, most of the images to betransmitted include an important portion and less important portions. Onthe basis of this fact, a method is proposed that reduces the subjectivedeterioration of an image quality by keeping the important portion freefrom the ill effects of deterioration.

For example, in systems such as video telephone systems, the image ofthe face region is so important that such systems have been directedtoward extracting the face region, preferentially allocating the amountof information transmitted and making the image quality of the faceregion better, aiming at improvement of the subjective image quality.

The prior art that is concerned with the present invention is disclosedin the publication "Color motion picture coding device featured bypreferentially processing a face image"(Hideyuki Ueno, Image Informationvol. 24, March 1992, pp. 29-35). The face-region extraction method ofthe prior art described in the publication will be explained as follows.

The prior art first determines the difference between two neighboringframes of the motion picture inputted to get the interframe differentialimage and then divides the interframe differential image intorectangular regions. In the respective rectangular regions, the methodmakes a histogram representing the number of pixels, in which theinterframe difference signal is larger than a certain threshold, on anH-axis. The method scans the histogram from the left to right, comparingits values with a certain threshold "th" to detect the left and rightends of the moving region.

The image screen is scanned from the upper left end until a first movingregion is found. This moving region has a coordinate on a V-axis, whichrepresents a position at the top of one's head. The image screen is thenscanned from the left and the right by a certain range of lower image todetermine the left and right ends of the moving region. And then, theoutermost positions of the moving region on the H-axis represent thecoordinates of the left and right ends of the face. The face length isdetermined by multiplying the face width by a predetermined coefficient"all". The prior art considers the thus determined image region as aface-region.

As mentioned above, the conventional feature-region extracting methodestimates a face-region from an interframe difference signalrepresenting the amount of motion. Consequently, if the image includesany other moving object than the face-region, the prior art method mayerroneously take the object as a face part. Furthermore, the method canroughly extract a face-region only in the form of a rectangle, whichcontour may be emphasized when the regions other than the face areadaptively processed with a low-pass filter. This may impair thesubjective image quality.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a feature-regionextraction method and the feature-region extraction circuit, which canextract a feature-region exactly along its contour without erroneouslyextracting any other moving object than the feature one, therebyallowing adaptive processing of portions other than the feature-regionwithout impairing the image quality and can extract a face-region, usingcolor difference components, regardless of the movement and location ofa person or a number of persons, improving the subjective image quality.

It is another object of the present invention to provide afeature-region extraction method whereby a small color difference regionincluding a feature color difference component of a feature-region islocated within a color difference coordinate system defined by two axesrepresenting two respective color difference signals and an imageregion, which has a color difference component in the small colordifference region mentioned above, is extracted as a feature-region; thesmall color difference region can take any desired form therebyeliminating the possibility of erroneously extracting any moving objectother than the feature-region; the feature-region can be exactlyextracted along its contour line to make the image more natural and tohave a subjectively improved quality and, therefore, can withstandadaptive processing of other regions with a low-pass filter; and,furthermore, only color difference signals are applied forfeature-region extraction to prevent erroneous extraction when theluminance level of the feature-region varies in space and in time.

It is another object of the present invention to provide afeature-region extraction method whereby a small color difference regionis a common region of two color difference signals binarized by the useof specified thresholds and a feature-region is easily extractedtherefrom.

It is another object of the present invention to provide afeature-region extraction method whereby a feature-region is extractedfrom a plurality of small color difference regions so as to furtherimprove the image quality.

It is another object of the present invention to provide afeature-region extraction method whereby a feature-region along itscontour line is extracted together with a dropout region enclosed withinthe feature-region contour line, so as to further improve the imagequality.

It is another object of the present invention to provide afeature-region extraction method whereby a feature-region is extractedfrom a common region of two color difference signals binarized by theuse of specified thresholds and dropout regions within thefeature-region are filled up by using detection patterns weighted bydistance so as to further improve the image quality.

It is another object of the present invention to provide afeature-region extraction method whereby in extracting a feature-regionby using means for extracting a feature-region and a memory means forstoring a feature-region extraction signal, a criterion is given ahysteresis characteristic depending upon whether a pixel backed by Lframes (L is a positive integer) and/or its neighbor pixel is afeature-region or not, making it possible to effectively prevent thefeature-region from being extracted with flickers and thereby to furtherimprove the subjective image quality.

It is another object of the present invention to provide afeature-region extraction method whereby a feature-region is extractedby using its characteristic, and feature-region extraction signals ofplural frames from a memory of M frames (M is a positive integer) areweighted for extracting a weighted feature-region so as to furtherimprove the image quality.

It is another object of the present invention to provide afeature-region extraction circuit whereby a feature-region extractioncircuit that includes a means for extracting a main portion of afeature-region and for filling a dropout region can extract afeature-region exactly along its contour line, and thereby adaptiveprocessing of an image region other than the featured one with alow-pass filter can be performed free from the image deterioration.

A feature-region extracting method, according to the present invention,can extract a feature-region by using a small color difference regionincluding a feature color difference component of a feature-regionwithin a color difference coordinate system having two axes representingtwo respective color difference signals, eliminating the possibility oferroneously extracting any other moving object than the feature-region.Furthermore, the method is capable of exactly extracting afeature-region along its contour, allowing adaptive processing with alow-pass filter to improve the subjective image quality withoutemphasizing the extracted region's contour. It binarizes two colordifference signals by use of specified thresholds to produce a commonregion and extracts therefrom a feature-region without erroneouslyextracting any other moving object than the feature-region.

According to the present invention, a feature-region can be extractedfrom a plurality of small color difference regions including a featurecolor difference component of a feature-region, and no other region thanthe featured one can be extracted even in the case when feature colordifference components are widely dispersed within a color differencecoordinate system having two axes representing two color differencesignals.

In addition, since a feature-region can be correctly extracted along itscontour line, it is possible to apply any adaptive processing such aspre-filter processing, which is a low-pass filter processing controlledby an extracted signal, or a quantizing control for roughening thestep-size of the quantization of a background region to improve thesubjective image quality without unnaturally emphasizing the contourline of the extracted feature-region.

To realize the above-mentioned purposes, the present invention was madeto provide:

(1) A feature-region extracting method which determines a small colordifference region including a feature color difference component of afeature-region within a color difference coordinate system defined bytwo axes of two color difference signals and which extracts an imageregion that has a color difference component within a small colordifference region mentioned above as a feature--region;

(2) A means to make a small color difference region be a common regionhaving two color difference signals binarized by certain thresholds andwhich extracts a feature-region therefrom;

(3) A means to extract a feature-region from a plurality of small colordifference regions (in addition to item (1) or (2)); or

(4) A means to extract a feature-region along its contour line togetherwith a dropout region enclosed within the contour line of thefeature-region as a feature region; or

(5) A means to binarize two color difference signals of an image bycertain thresholds, extract a feature-region from the common region ofthe binarized color difference signals and detect patterns for blankfilling which are suitably weighted according to the distance forfilling up the dropout region; or

(6) A means to provide a criterion with an hysteresis characteristicdepending upon whether a pixel backed by L frames (L is a positiveinteger) and/or an adjoining one relating to a feature-region or not anduses it when extracting a feature-region by using a means for extractingthe feature-region and a memory means for storing a feature-regionextraction signal; or

(7) A means to extract a feature-region by using its characteristic,generate output signals of feature-region extraction of frames from amemory of M frames (M is a positive integer) and make the framefeature-region extraction signals weighted to extract a weightedfeature-region; or

(8) A feature-region extraction circuit which comprises a feature-regionmain-portion extracting circuit for extracting an image region having afeature color difference component of a feature-region and a blankfilling circuit for filling up a dropout region which is sandwiched inbetween the feature-region and which has a different color differencecomponent other than the featured one; and further,

(9) A means to include the feature-region main-portion extractingcircuit which comprises a comparison circuit for comparing two colordifference signals with the upper limit threshold and the lower limitthreshold, and an AND gate circuit for determining the logical productof outputs of the comparison circuit and extracting a common region; andfurther,

(10) A means to include another feature-region main-portion extractingcircuit which comprises a first comparison circuit for comparing twocolor difference signals with a first upper limit threshold and a firstlower limit threshold, a first AND gate circuit for determining alogical product of outputs of the first comparison circuit andextracting a common region, a second comparison circuit for comparingtwo color difference signals to a second upper limit threshold and asecond lower limit threshold, a second AND gate circuit for determininga logical product of outputs of the second comparison circuit andextracting a common region and an OR gate circuit for determining alogical sum of the first comparison circuit's output and the secondcomparison circuit's output and extracting feature regions distributedamong a plurality of regions.

BRIEF DESCRIPTION OF DRAWINGS

FIGS. 1A, 1B and 1C show a diagram for explaining an interframedifference according to the prior art;

FIGS. 2A, and 2B are a diagram for explaining a conventional method forplotting a histogram;

FIGS. 3A, 3B and 3C are a diagram for explaining a conventionalface-region extraction;

FIG. 4 is a block diagram for explaining a feature-region extractioncircuit embodied in the present invention;

FIG. 5 is a diagram (1) for explaining the distribution of theskin-color of a person, according to the present invention;

FIG. 6 is a diagram (2) for explaining the distribution of theskin-color of a person, according to the present invention;

FIG. 7 is a diagram (3) for explaining the distribution of theskin-color of a person, according to the present invention;

FIG. 8 is a diagram showing an example of a circuit for extracting themain portion of a feature-region according to FIG. 4;

FIG. 9 is a diagram showing another example of a circuit for extractingthe main portion of a feature-region according to FIG. 4;

FIG. 10 is a diagram showing a further example of a circuit forextracting the main portion of a feature-region according to FIG. 4;

FIGS. 11A, 11B, 11C and 11D show a small region eliminating detectionpattern (1), according to the present invention;

FIGS. 12A, 12B, 12C and 12D show detection patterns (1) for blankfilling, according to the present invention;

FIGS. 13A, 13B, 13C and 13D show detection patterns (2) for blankfilling, according to the present invention;

FIGS. 14A, 14B, 14C and 14D show detection patterns (3) for blankfilling, according to the present invention;

FIGS. 15A, 15B, 15C and 15D show detection patterns (4) for blankfilling, according to the present invention;

FIG. 16 shows another example of a feature-region extraction circuit,according to the present invention;

FIGS. 17A, 17B, 17C and 17D show detection patterns (2) for eliminatinga small region, according to the present invention;

FIG. 18 shows detection patterns (3) for eliminating a small region,according to the present invention;

FIGS. 19A, 19B, 19C and 19D show a pattern (4) for judging a face-regionof a preceding frame for eliminating a small region according to thepresent invention;

FIG. 20 shows another example of a feature-region extraction circuitaccording to the present invention;

FIG. 21 shows weight coefficients according to the present invention;

FIG. 22, shows coefficients of a pre-filter according to the presentinvention; and

FIGS. 23A and 23B show variables of adaptive control according to thepresent invention.

PREFERRED EMBODIMENT OF THE INVENTION

FIGS. 1A to 1C are diagrams for explaining a conventional interframedifference signal, FIGS. 2A and 2B are diagrams for explaining aconventional histogram plotting method and FIGS. 3A to 3C are diagramsfor explaining a conventional face-region extracting method.

The prior art first determines the difference signal between twoneighboring frames (frame N shown in FIG. 1A and frame N+l shown in FIG.1B) of the motion picture inputted to get the interframe differentialimage shown in FIG. 1C and then divides the interframe differentialimage into rectangular regions as shown in FIG. 2A. In the respectiverectangular regions, the method makes a histogram representing thenumber of pixels, in which the interframe difference signal is largerthen a certain threshold, on an H-axis as shown in FIG. 2B. The methodscans the histogram from the left to right, comparing its values with acertain threshold "th" to detect the left and right ends of the movingregions.

As shown in FIG. 3A, the image screen is scanned from the upper left enduntil a first moving region is found. This moving region has acoordinate on a V-axis, which represents a position at the top of one'shead. The image screen is then scanned from the left to right by acertain range of lower image to determine the left and right ends of themoving region as shown in FIG. 3B. And then, the outermost positions ofthe moving region on the H-axis represent the coordinates of the leftand right ends of the face. The face length is determined by multiplyingthe face width by a predetermined coefficient "α" as shown in FIG. 3C.The prior art considers the thus determined image region as aface-region.

As mentioned above, the conventional feature-region extracting methodestimates a face-region from an interframe difference representing theamount of motion. Consequently, if the image includes any other movingobject than the face-region, the prior art method may erroneously takethe object as a face part. Furthermore, the method can roughly extract aface-region only in the form of a rectangle, which contour may beemphasized when the regions other than the face are adaptively processedwith a low-pass filter. This may impair the subjective image quality.

Referring, now to the accompanying drawings, preferred embodiments ofthe present invention will be described in detail as follows:

FIG. 4 is a block diagram for explaining an embodiment (1) of afeature-region extracting circuit according to the present invention,which comprises a feature-region main-portion extracting circuit 1, asmall region eliminating circuit 2 and a blank filling circuit 3. Thefeature-region main-portion extracting circuit 1 extracts an imageregion, including therein a feature color difference component of afeature-region and passes the extraction signal to a small regioneliminating circuit 2 whereby a small region is eliminated as a portionerroneously extracted by the effect of noise or the like. The outputsignal from the small region eliminating circuit 2 is passed to theblank filling circuit 3 which fills up a dropout region which has acolor difference component other than the featured one and then producesa feature-region extraction signal.

The present invention provides a method for extracting a face-region,utilizing the characteristic that skin-color components occupying mostof one's face-region as shown in FIG. 5, concentrate on a small colordifference region (hatched portion) within the color difference regiondefined by a coordinate system indicating a U-signal level on a verticalaxis and a V-signal level on a horizontal axis. As shown in FIG. 6, thesmall color difference region is made a common region by binarizing twocolor difference signals of an image through the specific thresholds andthen extracting a face-region therefrom. In short, this method is madeto extract a face-region on the basis that its feature (skin) color isapt to concentrate on a small color difference region (enclosed by, linesegments C_(HU), C_(LU), C_(HV), C_(LV)) within a coordinate systemindicating a U-signal level on a vertical axis and a V-signal level on ahorizontal axis.

Even if the skin color that occupies most of a human face-region isdistributed among a plurality of small color difference regions (e.g.the area A enclosed by line segments C_(HU1), C_(LU1), C_(HV1), C_(LV1)and an area B enclosed by line segments C_(HU2), C_(LU2), C_(HV2),C_(LV2) as shown in FIG. 7) within a coordinate system indicating aU-signal level on a vertical axis and a V-signal level on a horizontalaxis as shown in FIG. 7, it is possible to extract the face-regionexactly without error. Indeed, since human skin color components may bedistributed among a plurality of small color difference regions of,e.g., whitish, blackish or any other hue, it is desirable to extract aface-region by the method of the present invention.

FIG. 8 shows an example of the feature-region main-portion extractingcircuit shown in FIG. 4, which uses a memory 4 for implementing thefeature-region extracting method shown in FIG. 5.

The feature-region main-portion circuit 1 extracts a skin-color regionoccupying the major portion of one's face-region. This circuit 1 can becomposed of a memory 4 such as a random access memory, a read-onlymemory and so on. The memory 4 stores two color difference signals U andV which are connected to its addresses Al5 - A8 and A7 - A0respectively. In the memory space represented by 16-bit addresses, onearea corresponding to the hatched region of FIG. 5 is filled with thedigit 1 (logical one) and other regions are filled with the digit 0(logical zero). Accordingly, an output D of the memory 4 is 1 when thesignals U and V are signals of a skin-color region corresponding to thehatched region shown in FIG. 5, and changes to 0 when the signals U andV relate to a region other than the face's. The main portion of aface-region can be thus extracted. However, a face-region extracted atthis step lacks the regions such as the eyes, the eyebrows and the lips,differing in color from the skin-color portion. Furthermore, it may alsoinclude a region or regions other than the face-region, which wereerroneously extracted because of noise or the like.

FIG. 9 shows another example of a feature-region main-portion extractingcircuit shown in FIG. 4, which is composed of comparison circuits 5 to 8and an AND gate circuit 9 to implement the feature-region extractionmethod shown in FIG. 6.

The comparison circuits 5 to 8 compare two color difference signals (Uand V) with the upper and lower limit thresholds C_(HU), C_(LU), C_(HV),C_(LV). The comparison circuit 5 provides an output signal having thelogical value of 1 when U ≦C_(HU). Similarly, the comparison circuits 6to 8 provide output signals having a logical value of 1 respectivelywhen U ≧C_(LU), V≦C_(HV) and V ≦C_(LV). Output signals of thesecomparison circuits 5 to 8 are passed to the AND gate circuit 9 whichprovides a logical product of the output signals, extracting therefrom acommon region. Consequently, the output of the AND gate circuit includesthe logical value of 1 for the skin-color region enclosed by linesegments C_(HU), C_(LU), C_(HV), C_(LV) shown in FIG. 6. A main portionof a face-region can be thus extracted, but it lacks the regions such asthe eyes, the brows and the lips, differing in color from the skin-colorportion and may also include a region or regions other than theface-region, which were erroneously extracted because of a noise or thelike.

FIG. 10 illustrates a further example of a feature-region main-portionextracting circuit shown in FIG. 4, which is composed of comparisoncircuits 10 to 13 (for comparing with first upper and lower limitthresholds), a first AND gate circuit 14, comparison circuits 15 to 18(for comparison with second upper and lower limit thresholds), a secondAND gate, circuit 19 and an OR gate circuit 20 to implement thefeature-region extraction method shown in FIG. 7.

The comparison circuits 10 to 13 compare two color difference signals (Uand V) with upper and lower limit thresholds C_(HU2), C_(LU2), C_(HV2),C_(LV2). The comparison circuit 10 provides an output signal of alogical value of 1 when U≦C_(HU1). Similarly, the comparison circuits 11to 13 provide output signals of the logical value of 1 respectively atU≦C_(LU1), V≧C_(HV1) and V≦C_(LV1). The comparison circuits 15 to 18compare two color difference signals (U and V) with upper and lowerlimit thresholds C_(HU2), C_(LU2), C_(HV2), C_(LV2). The comparisoncircuit 15 provides an output signal of a logical value of 1 whenU≦C_(HU2). Similarly, the comparison circuits 16 to 18 provide outputsignals of the logical value of 1respectively at U a C_(LU2), V≦C_(LU2)and V≧C_(LV2). Output signals of the comparison circuits 10 to 13 arepassed to the first AND gate circuit 14 which provides a logical productof the output signals, extracting a common region. Output signals of thecomparison circuits 15 to 18 are passed to the second AND gate circuit19 which provides a logical product of the output signals, extractingtherefrom a common region.

The outputs of the AND gate circuits 14 and 19 are passed to the OR gatecircuit 20 which provides a logical sum of the outputs. Consequently,the output of the OR gate circuit 20 is a logical value of 1 for boththe feature-region A enclosed by line segments C_(HU1), C_(LU1), C_(HV1)and C_(LV1) and the feature-region B enclosed by line segments C_(HU2),C_(LU2) C_(HV2) and C_(LV2) as shown in FIG. 7. The main portion of aface-region can thus be extracted with no error even if an attribute ofa feature-region is distributed among a plurality of regions within acoordinate system indicating a U-signal level on a vertical axis and aV-signal level on a horizontal axis. However, the face-region extractedat this step lacks the regions such as the eyes, the eyebrows and thelips, differing in color from the face's skin-color portion and may alsoinclude a region or regions other than the face-region which wereerroneously extracted because of a noise or the like.

Therefore, the extraction signal from the feature-region main-portionextracting circuit 1 enters the small region-eliminating circuit 2(shown in FIG. 4) which identifies a small region erroneously detectedand eliminates it, as a region detected due to the effect of noise. Thiscircuit compares a feature-region signal with each of four detectionpatterns shown in FIGS. 11A to 11D and gives a logical value of 0 to thecenter pixel of a window of 3×3 pixels as a region other than the facewhen no match is found. A center pixel is judged to be a face-regionwhen a logical product of 6 pixels (∘:circle) shown in a circular formis 1 in each detection pattern.

The output signal from the small region eliminating circuit 2 is passedto the blank filling circuit 3 which compares its input signal with eachof four detection patterns shown in FIGS. 12A to 12D and gives a logicalvalue of 1 to a center pixel of a window of 13×13 pixels as aface-region when a match is found. A center pixel is judged as aface-region when both logical sums of 6 pixels (Δ:triangles) and 6pixels (⋄:diamonds) are 1 in each detection pattern. Detection patternsother than those shown in FIGS. 12A to 12D may be used if they have asimilar effect. For example, patterns shown in FIGS. 13A to 13B areapplicable as detection patterns. Furthermore, it is possible to useboth of these detection patterns (8 patterns) in such a way that thelogical value of 1 is given to a center pixel of a window when a signalpattern matches any one of the eight detection patterns.

The above-mentioned face-region extraction method can prevent erroneousextraction of any moving object other than the face-region and extractthe face-region exactly along its contour line, making it possible toadoptively process any other region than the face through theface-region extraction signal, e.g., with a low-pass filter withoutunnaturally emphasizing the contour lines of the face-region and therebynot impairing the subjective image quality.

Although the above-described embodiment of the present inventionperforms the extraction of a face-region, it is also capable ofextracting any feature-region other than the face by changing thresholdsfor color difference signals. The present invention also provides thepossibility to extract a feature-region by using a luminance signal incombination with color difference signals or by using color signals R,G, B and so on. In addition, windows having sizes other than 3×3 and13×13 are also applicable to performing the process to eliminateerroneously extracted small regions and to fill dropout regions with thesame effect. The present invention can also provide a feature-regionextraction method that does not include the small region eliminatingcircuit but assures a like effect.

As described above, the present invention provides a feature-regionextraction method whereby a small color difference region having afeature color difference component of a feature-region of an image islocated within a color difference coordinate system defined by two axesrepresenting two color difference signals and an image region that has acolor difference component within a small color difference regionmentioned above is then extracted as a feature-region. In case thefeature-region includes a portion having another color difference signalthan the featured one, the extracted feature-region includes thereindropout regions that, however, can be filled in to produce a completefeature-region.

The blank filling circuit 3 compares the feature extraction signal witheach of four detection patterns shown in FIGS. 14(a) to 14(d) andclassifies a center pixel of a window of 13×13 pixels as afeature-region and gives it a logical value of 1 when a match is found.In each detection pattern, a logical product of pixels shown in the formof a triangle (Δ) and a circle (∘) (logical product (1)), a logicalproduct of pixels shown in the form of a black triangle (▴) and a circle(∘) (logical product (2)), a logical product of pixels in the form of asquare (□) and a circle (∘) (logical product (3)) and a logical productof pixels shown in the form of a black square (▪) and a circle (∘)(logical product (4)) are first determined for each solid line, then alogical sum (1) of the logical products (1) and the logical products (2)of all lines is determined, a logical sum (2) of the logical products(3) and the logical products (4) of all lines is determined and alogical product of the logical sum (1) and the logical sum (2) isfinally calculated. A center pixel of the window of 13×13 pixels isjudged as a feature-region if the result of the final logical operationis 1.

Detection patterns other than those shown in FIGS. 14A to 14D are alsoused with the similar effect. For example, detection patterns shown inFIGS. 15A to 15D may be applied. A center pixel of a window of 13×13pixels is classified as a feature-region having a logical value of 1when the feature-region extraction signal matches any one of these fourdetection patterns. In each detection pattern, a logical product ofpixels shown in the form of a triangle (Δ) and a circle (∘) (logicalproduct (1)) and a logical product of pixels shown in the form of ablack triangle (▴) and a circle (∘) (logical product (2)) are firstdetermined for each solid line, and a logical sum of the logicalproducts (1) and the logical products (2) of all lines is thendetermined. A center pixel of the window of 13×13 pixels is judged as afeature-region if the result of the final logical operation is 1.

It is also possible to use a combination of both the detection patternsshown in FIGS. 14 and 15 in such a way that a center pixel of a windowof 13×13 pixels is judged as a feature-region and is given a logicalvalue of 1 if a match with any one of eight detection patterns is found.

FIG. 16 shows another example (embodiment 2) of a feature-regionextraction circuit according to the present invention, which comprises afeature-region main-portion extracting circuit 21, a small regioneliminating circuit 22, a blank filling circuit and a frame memory 24.The operation of this feature-region extraction circuit is as follows:

Similarly with the case of the embodiment of FIG. 4, the feature-regionmain-portion extracting circuit 21 extracts a skin-color region,occupying most of one's face-region. The extracted face-region, however,lacks different color regions such as the eyes, the eyebrows and thelips, and may include an region other than the face which waserroneously detected as a skin-color region because of a noise signal.Therefore, an extraction signal generated by the feature-regionmain-portion extracting circuit 21 is sent to the small regioneliminating circuit 22 whereby a small region, erroneously extracted dueto the effect of a noise signal, is eliminated from the extractionsignal. The output of the small region eliminating circuit is stored tothe frame memory 24 and a signal delayed by one frame is, returned tothe small region eliminating circuit 22.

The small region eliminating circuit 22 may use detection Patterns shownin FIGS. 17A to 17D in the case an extraction signal delayed by oneframe from the frame memory 24 is not a face-region and a detectionpattern shown in FIG. 18 in the case the extraction signal delayed byone frame from the frame memory 24 is a face-region. In each theextraction signal delayed by one frame is not a face-region, a presentextraction signal is compared with each of four detection patterns shownin FIGS. 17A to 17D and a center pixel of a window of 5×5 pixels isrecognized as a region other than the face and is given a logical valueof 0 when no match is found. In each of the detection patterns, a centerpixel is recognized as a face-region when a logical product of 15 pixelsshown in the form of a circle is 1. On the other hand, in the case theextraction signal delayed by one frame is a face-region, a presentextraction signal is compared with the detection pattern shown in FIG.18 and a center pixel of a window of 3×3 pixels is recognized as aregion other than the face and is given a logical value of 0 when nomatch is found. In this pattern, a center pixel is recognized as aface-region when a logical sum of 9 pixels shown in the form of a circleis 1.

Another example of detection patterns may be applicable in the smallregion eliminating circuit 22: The detection patterns shown in FIGS. 19Ato 19D are used for an extraction signal delayed by one frame from theframe memory 24. In the case they do not match any one of 4 detectionpatterns shown in FIGS. 19A to 19D and is not recognized as aface-region, the detection patterns shown in FIGS. 17A to 17D are usedas the detection patterns for the present extraction signal. On theother hand, in the case they match any one of 4 detection patterns shownin FIGS. 19A to 19D and is recognized as a face-region, the detectionpattern shown in FIG. 18, is used as the detection pattern for thepresent extraction signal. In each of the detection patterns of FIGS.19A to 19D, the extraction signal delayed by one frame is judged as aface-region when a logical product of 3 pixels shown in the form of acircle (∘) is 1 or 1 pixel shown in the form of a black circle () is 1.

In the case the extraction signal delayed by one frame is not aface-region, a present extraction signal is compared with each of 4detection patterns shown in FIGS. 17A to 17D and a center pixel of awindow of 5×5 pixels is given a logical value of 0 as a region otherthan the face when no match is found. In this case, each pattern is usedin such a way that the center pixel is recognized as the face-regionwhen a logical product of 15 pixels, shown in the form of a circle, is1.

On the other hand, in the case the extraction signal is a face-region, apresent extraction signal is compared with the detection pattern shownin FIG. 18 and a center pixel of a window of 3×3 pixels is given alogical value of 0 a region other than the face when no match is found.This detection pattern is used in such a way that the center pixel isrecognized as the face-region when a logical sum of 9 pixels, shown inthe form of a circle, is 1.

These detection patterns provide a relaxed face-region criterion for thepresent extraction signal when the extraction signal delayed by oneframe is a face-region. In short, a hysteresis characteristic is givento each face-region detection pattern for the present signal. This mayincrease the margin for noise to prevent the face-region from beingextracted with flickers. The output signal of the small regioneliminating circuit 22 enters the blank filling circuit 23 that,similarly to the case of the embodiment 1, fills in dropout regions (theeyes, the eyebrows, the lips and so on) of the face-region withdifferent colors other than the face's skin-color.

The above-mentioned face-region extraction method eliminates thepossibility of erroneously extracting any moving object other than theface-region. A face-region can be extracted exactly along its contourline, therefore the face-region extraction signal may withstand theadaptive processing of portions other than the face-region, for example,with a low-pass filter, assuring that the subjective image quality isnot impaired by unnaturally emphasized boundaries. According to thepresent invention, it is possible to prevent a face-region from beingdistorted with flickers that are easily detected by human eyes. Thisfeature may considerably improve the subjective image quality.

Although the above-described embodiment is used for extracting theface-region of an image, it can extract any kind of feature-region otherthan the face by changing the thresholds for color difference signals.The small region eliminating circuit can also use detection patternshaving other sizes than the windows of 3×3 and 5×5 pixels or may workwith other patterns having like effects. Similar effects can be obtainedby giving a hysteresis characteristic to thresholds of thefeature-region extraction circuit or to detection patterns of the blankfilling circuit.

According to the present invention, it is possible to obtain the sameeffects as those of the small region eliminating circuit and the blankfilling circuit even if either of these circuits is not provided.

The feature-region extraction method according to the present inventioncan extract a face-region with no flicker noise by giving a hysteresischaracteristic to a face-region criterion depending upon whether a pixelof a frame backed by L frames (L is a positive integer) and/or itsneighboring pixel is a feature-region or not.

FIG. 20 shows another example (embodiment 3) of a feature-regionextraction circuit according to the present invention wherein a framememory is indicated by numeral 31 and other components similar infunction to those of FIG. 4 are given the same numerals.

The frame memory 31 generates a series of feature-region extractionsignals of frame No. N (N is a positive integer) , N-1, N-2, . . . . N-M(M is a positive integer) through a memory of M frames (M is a positiveinteger). These feature-region extraction signals are weighted to gettherefrom feature-region extraction signals F_(N) weighted to beadoptively processed by pre-filtering or quantification control.

The weighted feature-region extraction signal F>, is expressed asfollows:

    F.sub.N =Σa.sub.N-r ·K.sub.r

where a_(N) indicates whether a feature-region extraction signal of aframe N relates to a face-region (a_(N) =1) or not (a_(N) =0: backgroundregion) and K_(r) is a weight coefficient of a frame No. (N-I).

If M=3, K_(r) can take, for example, any one of values (weightcoefficient) shown in FIG. 21. In the case of the example (1) all of K₀to K₃ are 1. In the case of the example (2) K₀ and K₃ are 1 and K₁ andK₂ are 2. In the case of the example (3), K₀ is 4, K₁ is 2, K₂ and K₃are 1.

When a two-dimensional pre-filter for processing a luminance signal, acolor difference signal has a coefficient shown in FIG. 22, a variableof adaptive control can take, e.g., values shown in FIG. 23A and 23B.FIGS. 23A and 23B show adaptive control variables respectively in thecase of not-weighted or weighted control. FIG. 23B shows a case that afeature-region extraction signal F_(N) is weighted as shown in theexample (1) of FIG. 21. In case of the feature-region extraction signalF_(N) being not-weighted, the coefficient P of the two-dimensionalpre-filter is equal to 1 and a variation ΔQ of a quantizing step size isequal to +10 when the feature-region extraction signal F_(N) is 0,whereas P=8 and ΔQ=0 when the signal F_(N) is 1. In case of thefeature-region extraction signal F_(N) being weighted, P=1 and ΔQ=+10are obtained when the feature-region extraction signal F_(N) is 0. P=2and ΔQ=+5 are obtained when the signal F_(N) is 1, P=4 and ΔQ=+2 whenthe signal F_(N) is 2, P=8 and ΔQ=0 when the signal F_(N) is 3, and P=8and ΔQ=0 when the signal F_(N) is 4.

According to the present invention, it is possible to use an outputsignal of the blank filling circuit 3 as a feature-region extractionsignal to be adoptively processed with a pre-filter or by quantizationcontrol. Furthermore, it is also possible to put an output signal fromthe blank filling circuit 3 into the frame memory 31 and use an outputsignal of the frame memory 31 as a feature-region extraction signal.

Application of the weighted feature-region extraction signal may notimpair the image quality even if the signal includes any erroneouslydetected portion. Noise may damage a feature-region extraction signal insuch a way that a certain portion may be erroneously extracted withdiscontinuity of time. In such case, if adaptive control variables shownin FIG. 23A are used, P and ΔQ can considerably vary depending uponwhether the feature-region extraction signal is 0 or 1, resulting inthat the signal includes flicker portions thus impairing the imagequality. On the contrary, if weighted adaptive control variables shownin FIG. 23B are used, the adaptive control variables vary smoothly asthe feature-region extraction signal changes from 0 to 4. This mayreduce the deterioration of the image quality.

The face-region extraction method according to the present inventioneliminates the possibility of erroneously extracting any moving objectother than the face. Since the face-region can be extracted exactlyalong its contour line, the face-region extraction signal may withstandthe adaptive processing of a background region with a low-pass filteror/and by quantization control for roughening the quantization step sizeof the background region/thereby assuring not to impair the subjectiveimage quality with an unnaturally emphasized boundary.

Although the above-described embodiment is used for extracting theface-region of an image, it can extract any kind of feature-region otherthan a face by changing the thresholds for color difference signals. Itis also possible to perform a feature-region extraction by usingluminance signals in combination with color difference signals or byusing color signals such as R, G, B and so on. The windows withdifferent sizes other than 3×3 and 13×13 pixels may be applicable withlike effects for small region elimination and blank filling processing.The feature-region extracting method, according to the presentinvention, can realize the fine extraction of a feature-region on animage without a small region eliminating circuit.

As this invention may be embodied in several forms without departingfrom the spirit of essential characteristics thereof, the presentembodiment is therefore illustrative and not restrictive, since thescope of the invention is defined by the appended claims rather than bythe description preceding them, and all changes that fall within metesand bounds of the claims, or equivalents of such metes and bounds aretherefore intended to be embraced by the claims.

I claim:
 1. A method of extracting a feature-region from a color imagefor purposes of image transmission in a transmission system havingfinite transmission capacity, comprising the steps of:defining a colordifference coordinate system having two axes, and representing eachpixel of said color image by two respective color difference signals insaid system; identifying at least one region of contiguous pixels, basedon whether their color difference signal values fall within apredetermined value range, as a small color difference region definedwithin said color difference coordinate system; extracting an imageregion having color difference signals within said color differenceregion as the feature-region to which to apply preferential treatment,compared to a remaining portion of said color image; and allocating agreater part of said finite information transmission capacity to saidextracted feature-region as said preferential treatment, and a remainingpart of said capacity to a non-extracted portion of said color image. 2.A method according to claim 1 which predetermines a small colordifference region including feature color difference signals of afeature-region within a color difference coordinate system having twoaxes, each axis representing a respective color difference signal,andextracts, as the feature-region, an image region having a colordifference component in the predetermined small color difference regionmentioned above.
 3. A method of extracting a feature-region from a colorimage, according to claim 2,wherein the predetermined small colordifference region is a common region of two color difference signals,which are respectively binary signals, digitized by respective specifiedthresholds and wherein the feature-region is extracted therefrom.
 4. Amethod according to claim 1, which extracts a feature-region along itscontour line, together with dropout regions, which do not satisfy saidpredetermined criteria, but which are enclosed within the featurecontour line of the feature-region.
 5. A method according to claim 3,wherein any dropout regions within the feature-region are filled in byapplying predetermined detection patterns which consist of nearbypixels, and are weighted by distance.
 6. A method of extracting afeature-region from a color image, according to claim 2,wherein thefeature-region is extracted from a plurality of predetermined smallcolor difference regions.
 7. A method of extracting a feature-regionfrom a color image, according to claim 3,wherein the feature-region isextracted from a plurality of predetermined small color differenceregions.